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A computer netvrorit based system and method for collecting and reporting data, 
preferably market research data in real-time. Real-time feedback enables timely 
response to emerging issues, enhances training/feedback for support staff and 
gives the ability to promptly see problem areas through the users' eyes. The 
present invention collects and reports market research data. A software 
implemented data collection tool (16) deploys to a dient (18) over a computer 
network (2) to elicit data from the user. The collection tool (16) returns the elicited 
data to a research database (6), which stores the elicited data. The research 
database (6) is accessible to one or more designated persons, typically a 
commissioner of the research vrark and/or someone with their authority. 
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Field of the Invgntinn 

The present invendoo relates to computerized market research tools, and in paitiadar' 
to a computer network based system and method for coUcctins and rqjorting data, prefciBbly 
10 msrket research data, such as survey data. 

Background of tht- hVPinPIl ' 
As computer network technology, in particular Internet technology, has rapidly 
developed in recent years and businesses and other organizations have found or been 
presented with more elective ways to take advantage of this Internet technology, the World 
15 Wide Web has taken on ever increasing importance in the world of commerce. More and 
more organizatioiis are using the Internet as a primary ch«"T<gl of communication, 
supplementing or even replacing traditional lines of communication such as face-to-foce 
meetings, mail, fex ami phone. A recent smdy by 
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Forrestcr Research, Inc. estimates that Intcma commerce wilt grow forty fold over a period of 
five years, from USS8 biJlion in 1997 to USS327 billion in goods and services by the year 2002. 

Even now the lotemet ptsys a very sigmScani role as a distribution channel, a point of 
contact with customers, a marketing and advertisii^ medium and a channel for services and 
support, and the significance of this role will only increase in the future. Concordant with this 
growing significance of commerce over the Imemet, in order that organizations can maintain and 
improve on the standards of information they share via the Internet, there is a rapidly growing 
need for feedback on the organizations' Internet presence, and thus a need for tools which can 
quickly and eScieotiy collect and repon relcvam market research data characterizing and 
evaluating the organizations' Internet efforts. 

Traditionally, market research has been conducted by way of a survey completed during 
an interview between a market researcher and a consumer, cither &ce-to-face or over the 
telephone. As an alternative, the survey may be m^led on paper or on computa diskette to the 
consuma for them to complete and return. The survey contains a scries of questions designed to 
elidt relevant mforaiation from ihe consumer, which can be collated and analyzed with answers to 
the survey &Qm a number of other consumers to provide a characterization and/or an evahwtion 
of panicuiar facets of the organization, services or products of the entity for whom the survey is 
being conducted. 

Many companies or businesses initially developed on line survey tools in-house. More 
20 recently, computerized survey tools have been provided, whereby survey or other market research 
information can be elicited over the Internet. One example of such a tool is 'SurveySez', which 
allows an organization to create their owm HTML-based survey as a page or pages hosted on the 
SurveySez Internet Web Site. The organization then indudes a link to the survey on their own 
Web site. 

25 A second example is 'SurveySite', which uses a pop-up invitation implemented in lava 

which acts as a link to an HTML survey hosted at a separate site. A report is generated and 
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defivered to the organizatioo commisstooing the research at the end of the survey's life. e.g. 

after a pre-determined number of surveys have been responded to. 

Another example is 'CLTResearch'. offering a survey in flat HTML. Similaiiy to 

'SurveySite' a repon is generated at the condusioo of the complete survey period. Vet 
.5 another example ofan HTML survey with a repon generated on completion is TCBAP". In the 

latter example, the respondents to the survey may either be visiton to a particular Web Site, 

as in the above examples, or previously registered partidpants. 

A final example, which employs a very different methodology to the previous 

examples, is UdevantKnowiedge'/Mielsen Croup. A group of people (called a web pand or 
10 research pand) agree in advance to have their Internet usage monitored, and then periodically 

a log of which Web Sites these people have visited is uploaded to a centra) collection point, 

and a repon is generated. 

Summary of the Invmtion 
The present invention provides a computer network based system and method for 
15 collecting and reporting data, preferably market research data in real-time. Whatever the 
^plication to which the invention is applied, paniculariy where it is maricct research oricmed, 
the real-time feedback it provides can enable timely response to emerging issues, can enhance 
training/feedback for suppon staff and can give the ability to promptly see problem areas 
through the users' eyes. 

20 From one aspect, the invention provides a computer network based, preferably 

Internet based, system and method for collecting and reporting data, preferably market 
research data A software implemented data collection tool is deployed to a client on the 
computer network (for example a Web browser) to elidt data from a user accessing the 
computer network through that dient. The collection tool returns the clidted data to a 

25 research database, which stores the elidted data. The research database is made accessible to 
one or more designated persons, typically a commissioner of the research work and/or 
someone with their authority. 
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The data cofleaion tool can be deployed to successive users of the computer network, 
accessing the network via the same or. more typically, a different dient. Eadi time the 
collection tool is deployed and data returned, the research database is updated, and thus 
updated, real time results are made accessible to the designated person(s). 
5 Preferably, the deployment of the data collection tool is initiated by the user visiting a 

particular pre-defined trigger page of information on the computer network, or any one of a 
number of such presjcBned trigger pages. Typically, each trigger page wUi be assodated with 
a panicuiar infonnation site of the organization commissioning the research. For example, 
where the system is implemented on the Imemet, each trigger page may be a page of thdr 
10 Web site. If there are two or more trigger pages at an information site, the system is 
preferably comrolled so as to suppress deployment of the data collection tool more than once 
during any single visit to the infonnation site by a user. For instance, the dau coilecuon tool 
can fiinction to detect whether or not it has previously been run during the cunent browser 
session in order to prevent multiple deployments, 
13 In some embodiments of the invention, further control is exerdsed over the 

deployment of the data collection tool. For instance, a first stage in the deployment of the 
collection tool may be the issue of an invitaUon to the user, for example giving them the 
option to dcdine to take the survey or other data collection means if they wish. If desired. 
. any sudi decision to decline to take a survey may be recorded in the research database. A 
20 further possibility is to control the frequency at which the data collection tool is deployed. 
That is to say. the tool need not be deployed eadt time a user visits a trigger page, but rather 
can be presented only to every n* user to visit the page, or to a random one of every o users 
(where n is any real integer greater than I) and/or only after lapse of a predciennined time 
imerval foDowing a preceding deployment of the tool. Yet a further possibility, where the 
23 users of the computer network are identified with a user name or other identifier sudi as 
machine or source IP address, is to place limitations on the identity of the users to whom the 
data collection tool b presented. For instance, it may be desirable to exclude certain 
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namedAidentified users from the market research or other dau collection process, for example 
employees of the research commissioner. In addiuon or alternatively, particularty on 
computer networks such as the Internet where individual users do not necessarily require 
positive identification as a condition of access, spedfic network addresses (e.g. IP addresses 
5 on the Internet) can be blocked to provide a similar control over the persons to whom the data 
collection tool is presented. 

Once the data collection tool has been deployed, the system preferably has the ability 
to dday the actual commencement of data didtation. For example, the data collection tool 
may have the ability to linger within the diem for a period of time after its deployment, during 
10 which time the cUeni device can be operated to perform other functions. For example, it may 
be desirable to deploy the data collection tool when a user first arrives at an information site, 
but to dday didtation of the market research data, or other data, until after the user has had 
an opportunity to explore the site more fully. 

In a typical implementation of the invention, the data collection tool will provide a 
1 5 survey or other data form, presenting the user with a series of questions to be answered. For 
some questions the user may be presemed with a series of possible responses frtjm which they 
must choose one. or in some cases one or more. Other questions may require a response by 
way of a vahje on a continuous scale, aiwl another option is to allow the user to enter free text 
answers. Where there is some control placed on the response to a question, such that the 
20 response is predictable, the system may be designed to use the response to that question to 
dedde whether a subsequent question in the survey be used. Once all of the answen to the 
survey have been provided, the data coileaion tool ddivers the answers to the research 
database for access by the reporting tool at the report site. In addition to the answers 
themsdvcs. the data collection tool may also collect and return other types of informatioa for 
25 example infonnation indicating the state of complerion of a survey, or other similar sample 
tnddcncc imortnation. 
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In some onbodimenu of the system, the data coileoion tool coaducu one or more 
analyses of the data to produce one or more report results, which arc stored in the database. 
The analysis or analyses may be rdaiivdy simple, for example calculating occurretjces of a 
particular class of answer as a pcrcemage of total answers to a question, or relatively 
5 complex, for example deriving a report result based on answers to a number of different 
question. The data collection tool may also directly store the raw information eKdted as 
report results in the database, for example answers to one of more of the qucstioiu presented 
in a survey may be stored without further analysis. Furthermore, in preferred embodhnems, 
the at least some report results are also available in an interactive query format, allowing 
!0 someone accessing the results via the repon site to spcdfy an analysis based on a combination 
of any two, or possibly more than two. of the non-free text questions of a survey. 

The research database is preferably accessible via a report site on the computer 
network which, if desired, can provide secure access to the database. For instance, the repon 
site may be set up so that a pre-arranged password is required before the repon site enables 
15 the research database to be accessed, the password only being made available to pre- 
authorized designated persons. In this way, the results of the market research, or other data 
collection process, can be made avaibble only to the desired parties, for example the 
commissioner of the research, or persons with their express consent. 

The report site preferably also provides a user friendly front end to the database, and 
20 comprises or works in conjunction with a software implemented report tool to extract relevant 
report resulu from the database and present them to the designated person(s) as an 
appropriately fonnatted report. Typically, the results will be presented graphically as bar 
charts, pic charts, tables or the like, but any "one of a large number of other graphical or 
textual formats are possible. It is particularly prcfen-ed that the report tool has the capability 
25 to manipulate the report results and/or carry out one or more analyses of them, in order to 
present the designated person(s) with a variety of differently focused reports. For example. 
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the system may allow the designated person to selea one of a number of focused reportv or 
even to define the focus of the report themseWes. 

Advantageously, the data collection tool can be deploy &om a server (referred to 
hereinafter a "survey server*) associated with the computer nerwork, other than a server 
5 (referred to hereinafter as an "information server") on which the information site containing 
the trigger page or pages is located. This arrangement, unlike prior survey systems, can avoid 
placing additioiul load on the information server. The research datnbnw' is also preferably 
resident on a server (referred to hereinafter as a "report server") other than the information 
server, as is the report site and report tool, The research database, the report site and the 
10 report tool are preferably resident on a single report server, which conveniently may be the 
same server as the survey server from which the data collection toot is deployed. 

la a development of the present invention, the dau collecting and reporting system 
elicits information from the users of a plurality of distinct information sites on the computer 
network, which may be resident on the same or different information servers. For each 
15 information site the same data collection tool and/or the same report tool may be employed. 
Alternatively; custom versions of one or both of these tools may be provided for use with one 
or more of the information sites served. If different, customized collection and repon tools 
are used for respective ones of the informauon sites served by the system, they are preferably 
each based on a respecuve one of a master data collection tool and a master repon tool, which 
20 provide common functions generally required by most, if not all market research applications 
or. if applicable, other data collection applications to which the system applies. In this way, 
the effort required to provide a custom dau collection and/or report tool for any particular 
application can be minimized. 

Where multiple information sites are served, all of the data elicited may be stored in a 
25 single research database, but access to the data is preferably controlled so that the report 
results associated with the distinct information sites are only accessible to the respective 
designated person or persons. For example, each designated person may be given a unique 
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access name and con^sponding password which they use to enter the report site, the report 
site and report tool utilizing the unique access name, if verified with the corresponding 
password, lo determine which repon results should be made accessible to a particular 
designated person. 
^ Brief Description of ^h*? fPril'^^rigfi 

The foregoing aspects and many of the attendant advantages of this invention will 
become more readily appreciated as the same becomes better understood by reference to the 
foUowing detailed description, when taken in conjunction with the accompanying drawings, 
wherein: 

10 FIGURE 1 schematically illustrates a market research data collection and reporting 

system that operates in acconiance with an embodiment of the present invention: 

FIGURE 2 is a block diagram showing the stnicture of the diem side component of 
the data collection tool of the system of FIGURE I; 

FIGURE 3 schematically iUustrates the procedure by which the system of RGURE 1 
15 collects data; 

FIGURES 4 to 17 show screen shots from an exemplary survey conducted over the 
tmemet using the inventive system; and 

FIGURES 1 8 to 28 show pages from an exemplary repon site according to the present 
invention. 

^° P«8il«l Pe^crintion of the Pr^f^rf ^ EmfaQdim.^t 

The present invention is a computer network based system and method for collecting 
and reporting data, preferably market research data in real time. Cunently, the primary 
application of the invention is seen to be in the field of gathering customer feedback regarding 
the quality of services, infortnation and prtKlucu thai are being provided by an organization 

25 doing business on the World Wide Web, ar«l the foDowing description of an exemplary 
illustrates such an application. However, the invention is also applicable to gathering other 
types of data, whether market rcsean:h data or other informational data, in any of a number of 
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computer network environments. In this sense, it can be considered as a general purpose 
market research tool. 

Referring to FIGURE I, a data collection and reponins system is implemented on the 
Worid Wide Web. data exchange between various components of the system being effected 
5 over the Internet 2, Visitors to a specific information site 4 on the Web. typically a Web site 
of the research commissioner, are presented with a survey inchtding a number of questions. 
The answers to these questions given by the visitors are stored in a database 6 and made 
immediately accessible to a designated person or persons, typically the research commissioner, 
by way of one or more user friendly survey reports available on-line over the Imernet 2 at a 
10 report Web site 8. The survey remains active for a finite period defined by the number of 
visiton to take the survey, an overall time duration, or a combination of the two. At the end 
of the complete survey period, the results remain accessible to the designated pcrson(s) for a 
further pre-defined post-survey time period. At any time during pendency of the survey or 
during the post-survey period, the survey results can be down-loaded from the survey 
15 database 6. in order that a permanem record may be kept by the research commissioner. 
Additionally, any of the graphical reports at the report site 8 can be captured using common 
methods for HTML capture and reposted to any other Web site (for example in onJer that the 
research commissioners intranet can be used to share the infonnation within an organization) 
or the graphics can be incorporated into any compatible tool for presentation. The main 
20 elements of a cunenily preferred embodiment of the system are: (a) a data coilecdon tool, 
having both server side and diem side components, for presenting the user with the survey 
and coflecdng their answen as survey data; (b) a survey database 6 for storing the survey 
data, either in raw form or foUowing one or more analyses or manipulations, as survey results, 
and whid) also contains survey configurauon data and survey question text; (c) a report site 8 
25 through whidi the designated person can access the survey results; (d) a report tool ! 0 whidi 
b implemented within the report site 8 to present the designated person with the survey results 
in the form of one or more focused survey reports; and (c) a trigger, resident on a trigger page 
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12 of an infonnation site 4. typically a Web page of the research commUsioner, to initiaie the 
survey when the visitor retjucsts that trigger page 12. 

jWvaruageoujIy. a single survey server 14 operating ihe Inventive sysiem may be used 
to simultaneously condua more than one survey, each survey being associated with the same 
5 or a different Web site or page on the World Wide Web. For each Web site or page the same 
fundamemat dau collection tool is employed, although an expanded survey (tnchiding 
additional questions) or otherwise customized survey (e.g. inchiding revised questions and/or 
graphic images) may be used. Where multiple surveys are being conduaed, the survey results 
from each survey are stored in the database 6 in a manner identifying them with thdr 
1 0 respective survey, in order that the rtpon site 8 and report tool 1 0 can distinguish them. !t is 
also akely tiiat the designated per3on{3) will be different for the different surveys. 
Consequently, each designated person is assigned a unique user name, verified by a password, 
for accessing the report site 8 and hence the survey results stored in the survey database 6. In 
this way it is possible to restrict access to xht survey results associated with particular surveys 
15 only to respective authorized designated persons. 

b the preferred implementation of the data collection tool, there is a capability to 
suppress presentation of the survey to the visitor in cenain circumstances. More particularly, 
when initiaih- deployed the survey applet 16 conducts a vaUdation operation to determine 
whether to proceed with the survey, in which it is determined, for example: (a) whether the 
20 implementation of the survey is supported by the visitor's Web browser 1 8; (b) whether the 
survey has been previously executed in the current browser session; and (c) whether the IP 
address from which the visitor is accessing the network is on a list of addresses excluded from 
participation in the survey. If these and/or other validation tests are passed, that Is to say, for 
example, the visitor's browser 18 will support the survey, the survey has not been previously 
25 executed and the visitor is not exchided, the applet 16 proceeds to the next step in the 
procedure. If any one of the tests is failed, the survey applet 1 6 exits without ever presenting 
itself to the user. 
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Also in this preferred iraplemcntation, the frequency at which the survey is presented 
to visiton to the trigger page 12 can be controUed. Thai is to say the survey need not be 
presented to each visitor who passes the above vaUdaiion operation. This frequency control a 
implemented by setting an invitation frequency (/) within the survey applet 16. The vahje/is 
5 M integer which determines the probability thai a user will be presented with the survey. If/ 
is set to I. every visitor who passes the validation will be presented with the survey, whereas 
integers greater than I will reduce the probability thai a given visitor will be prescmed with a 
survey. For insumce, the survey applet 16 can be configured to randomly present the survey 
to one out of every n visiton. where / - n. In the current implementation, a visitor is 
10 presented with an invitation if the results of QooriR x/) equals 0. where /? is a pseudo random 
number between 0 and I. The floorQ function truncates the product of Rxf, that is to say 
subtracts Oie fractional pan of the product, to give an integer value. In the preferred 
Implementation, the vahie/can be changed during the course of a survey, allowing control of 
the period during which data collection is completed, providing the unique ability to ensure 
1 5 coilecaon of a representative sample of site traffic. 

The survey applet 16 also has the capability to provide a further front end filter to the 
survey. If desired, rather tiian launching straight in to the full survey itself, the visitor can be 
presented with an invitation to take the survey. This invitation can give the visitor the 
opportunity to decline to take the survey (such a decision to decline itself being recorded as 
20 the survey data), can be used to "screen" for a specific type of visitor (e,g. visitors with a 
specific connection speed or other screening information), and can also be conveniendy used 
to offer the visitor an incentive for completing the survey. 

Whatever front end filters are used, if and when the fiill survey is launched, the survey 
applet 16 presents the visitor with a "STAKT SURVEY" icon, along with instructions on the 
25 completion of the survey and, if desired, a suggestion to them that they complete tiieir 
business at the Web site before taking the survey, in order tiiat their responses might be based 
on a more complete review of the Web site. The "START SURVEY" icon exists in its own 
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window separate from the main browser window, so it may be minimized or lowered while the 
visitor continues to browse the information Web site 4. When the user is ready to commence 
answering the survey questions, they simply activate the "START SURVEY" icon as they 
would any other icon (eg. by clicking on ii with their mouse), and they are presented with the 
S first question. 

Considering the data coUection tool in more deuiil. As mentioned above, the data 
. collection tool comprises both server side and client side components, which communicate via 
the Internet 2 using the standard HTTP prowcol. The server side components are resident 
on, and are executed by the survey server 14. They are currentiy implemented as a series of 
10 servlets, which run under the JavaWebServer I.I environment. They could, however, be 
implcmemed to run under any HTTP server that suppons Java servlets or other equivalent, 
applets. Currently there are four server side components. 

First, there is a survey servlet 20 which, at the request of the cliem side component 16. 
extracts the appropriate survey configuration data from the survey database 6 and posts it to 
1 5 dicnt side component 16. Where the system is set up to handle multiple surveys, in order that 
the survey servlet 20 can extraa and post the correct survey configuration data for tiie survey 
that is to be activated, the survey servlet 20 is first passed a pre-assigned. unique survey job 
code by the cUent side component 16. The client side component 16 itself obtains the unique 
survey job code from the trigger on the trigger page 12. the code being one component of the 
20 trigger. Using the unique survey job code, the survey servlet 20 retrieves the appropriate 
configuration data from tiw survey database 6. puts it in name-value pair format and posu it 
(0 the client side component 16. In currently preferred 'embodiments, the configuration data 
tnchides the following dcmcnu: 

•Survey class name' - this is a class file that determines the question set 
25 for each survey. Each different survey may be associated with its own unique 

survey class, in which case the text of the questions, which make up the survey 
can be coded into the class. However. In a preferred tmpicmentatioo a generic 



survey class is used for a number of different surveys, the generic class 
configuring itself for any particular survey by retrieving the respeaive question 
set from the survey database. With this latter implementation, the survey class 
name will be the same for all surveys sharing the generic survey class. 

"Survey Job Code' - this is the key to retrieving the correct survey 
configuration for a particular survey. It is passed to the client side component 
16 of the dm collection tool as a parameter from the uigger. 

"Question Image (banner)" - this is an image that is displayed at the top 
of each question frame and may, for example, contain the name or other 
identifying symbol of the research commissioner. 

"Start Image (logo)" - this is an image that appcan vmh the 'START 
SURVEY" icon. 

'Question Color" - in the exemplary implementation, as will be 
explained further below, the visitor b presented the survey questions in a 
question frame which contains two panels: (i) a question panel in which tiie 
question text is displayed; and Ci) a response panel in which the visitor makes 
their response. The "Question Color" parameter determines the background 
color of the question panel. 

"Re^nse Color" - this parameter determines the background color of 
the response panel in the question frame. 

"Frequency* - this is the vahie / which determines the probability that a 
user will be presented with an invitation. 

"Blocked IP" - this is a string containing a comma separated list of IP 
network addresses for which the applet is to be blocked, i.e. addresses at which 
the survey will not be presented to visitors. 



-1+ 



"Invitation Text" - this find cJement of the configuration data contiina 
text that appears in an invitation window, inviting the visitor to tike the 
survey. 

The second server side component of the data collection tool is s dau grabber servlet 
5 22, This servlet accepts the survey data from the diem side component 16 and updates the 
survey database 6, which is preferably a relational database, with this data. More specifically, 
the client side component 16 sends the survey data to ibe survey server 14 over the Intentet 2 
as survey data post requests. The data is posted in name-vahie pair format. The data grabber 
seivlel 22 accepts the HTT? request, extracts the survey data and posts it to the survey 
10 database 6 using, in the ctirrmtly preferred embodiment, the JDBC API (a standard feature of 
JDKl.l). 

Third, there is a question servlet 24 which accepts question text requests from the 
client side component 16. extracts the appropriate question text from the survey database 6 
and returns the question text to the client side component 16. The question data is returned to 

15 the dient side component 16 in string format via an HTTP response. The client side 
component 1 6 parses the string formatted data and uses it to initialize Java question objects. 
The question text itself is designed vrtth a view to the particular survey being undertaken. The 
currently envisaged application for which the exemplary system according to the Invention is 
designed is the gathering of customer feedback regarding the quality of services, information, 

20 and products that are being provided by an organization doing business on ihe WoHd Wide 
Web. Suitable standard, core questions for this application are given in Appendix I, although 
in any particular case it may be desirable to add further, custom questions, or to modify some 
of the standard questions to provide the desired focus for the market research being 
undertaken. 

25 As an alternative to returning the question data in string format, it may be returned 

using object serialization if the programming environment allows, for example if the survey 
applet is running under JDKl. 1. Object serialization allows an object instance to be encoded 
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into a byte stream, which can then be written to a file or network socket. The byte stream can 
be subsequently read from the file, or the other end of the network socket, and used to 
instantiate another copy of the original objea. With this objea scrializauon approach it 
becomes possible to simplify both the question servlet 24 and the client side component 16 
5 because they are no longer required to conven tpiestion dau to and from string format. 

The final server side component of the data coUection tool is a counter servlet 26 
which accepts data packages from the client side component 16 containing survey progress 
data, that is data indicating the progress of die survey, rather than data representing the survey 
snswen themselves. It is envisaged that this data couid be used to detect when a visitor 
10 reaches particular milestones in the completion of the survey, and would enable analysis of the 
actions of visitors who sian the survey but drop out prior to completion. 

The survey server 14 also holds a copy of the client side component 16 of the data 
coUecdon tool, which is deployed to. and executed within the visitor's Web browser 18. In 
this exemplary embodiment, the client side component 16 is implemented as an applet written 
1 5 in the Java language and will be referred to a the survey applet 1 6. It prtivides the imerface 
with the visitor through which the survey data is collected. More particularly, the survey 
applet 16, once downloaded and executed within the visitor's browser IS, presents the visitor 
wih a series of questions, one ai a time, in a window separate from the browser 13. The 
visitor responds to each question in turn and once they have coramiitcd a response, the applet 
20 16 does not allow the response to be changed. The applet 16 initially records each response 
on the ciiem side, and then when all of the survey questions have been answered, the 
responses survey data) are posted to the survey server 14. 

This approach has a number of advantages over prior survey techniques. Firstly, the 
ability to prevent answers being changed once made is generally desirable, and using a Java 
25 applet, or an equivalent applet, enables this to be achieved whilst keeping the number of 
transactions with the survey server 14 to a minimum. Thus the survey applet 16 collects all 
user responses, question by question, and posts them to the survey server 14 in a single 
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transaction, whereas the known HTML implemented survey toots would require cither a 
transaction after each question response or putting all questions on a single form. In the first 
case the user may experience frustrating delays between document downloads which would 
tend to increase the survey drop out rate, thereby jeopardizing the confidence intervals and/or 
5 statistical validity of the survey results. The latter case would require HTML and server side 
CGI code, which would be very difficult to maintain. In dther of these HTML 
implementations, the user can easily go back to change previous responses, The ability of the 
present invention to control these aspects of behavior on the part of the survey tool and the 
actual survey respondent hdps to ensure the validity of the methodology ami the integrity of 

10 the data collected. This is an importam methodological improvement offerKi by the present 
invention over other knovm survey tools. 

A fijrther advantage is that since the server applet 1 6 runs in a separate window, it can 
be minimized allowing the user to explore the Web site 4 they arc visiting before starting the 
survey. This may increase the likelihood of a more informed evaluation of the site by the 

1 5 visitor. The use of the Java applet also provides a more versatile and robust user interface, for 
example it is not affected by connectivity problems once up ami running, and offers a superior 
development environment that results in both source code that is easier to debug, maintain, 
and modify incrementally, and research results that are methodologically more reliable than in 
other known on-line survey tools, 

20 The structure of the survey servlet 16 will now be discussed in greater detail, with 

reference io FIGURE 2, which is an objea diagram for a preferred implcmeniaiion of the 
survey servlet. 

The APPLET LOADER 30 u the component rnitiaily downloaded from the survey 
server 14 by the visitor's browser 18 when the visitor requests the trigger page 12, containing 
25 the trigger, from the information server 4. Its ftmctions are to download from the survey 
server the necessary Java dasscs. to present the invitation lo take the survey to the visitor, and 
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to instantiate the CLmm SPECIFIC SURVEY 32 and call the doSt>rvcyO method, when the 
INVITER 34 indicates a positive answer by a visitor to the invitation. 

CLIENT SPECinC SURVEY 32. for whidi the WEBMETRICS class 36 is a 
superdass. overrides the doSiirveyQ method, and instantiates the QUESTION FRAME 38. 
5 the concrete QUESTION objects 40 and CGI POST 42. In addition it passes, in turn, a 
reference to eadi QUESTION object 40 to the QUESTION FRAME showQ method, to cause 
each question of the survey to be presented in turn. If desired, the survey may be configured 
so that the presentation of cenain QUESTION objects 40 is dependent on an answer to a 
previous question. For example, one might only wish to ask. the visitor what ii^dustry sector 
10 they work in if they have already indicated that they were visiting the information site of the 
research commissioner for business purposes. Skip logic such as this, and other similar 
question controls are not available in conventional HTML survey tools. 

As each question is answered, CLIENT SPECIFIC SURVEY 32 calls the 
getResponseO method of QUESTION 40 and the addPosiVaiueO method of CGI POST 42 
1 5 to retrieve the visitor's response to the question and update CGI POST 42 with the response. 
Fmally, once all of the survey questions have been answered. CLIENT SPECIFIC SURVEY 
32 calls the CGI ?0%1 postQ method to return the answers to the survey server 14. 

QUESTION FRAME 38 contains QUESTION 40, and displays QUESTION 40 when 
its shoKvO method is called by CLIENT SPECIFIC SURVEY 32. QUESTION FRAME 38 
20 also handles action evcnu from NEXT BUTTON 44, which accepts the visitor's indication 
that they have answered the current question. A NEXT BUTTON event handler within 
QUESTION FRAME 38 calls the' oonResponseQ method of QUESTION 40 and updates 
PROGRESS METER 46 as eadj question is answered. If the visitor attempu to move on to 
the next question without having competed the answer to the current question, the NEXT 
25 BUTTON handler causes an appropriate error message to be displayed to the visitor, or takes 
other corrective action if the survey methodology so indicates. 



QUESTION 40 coniains a QUESTION PA^fEL 48 in which the question text is 
presented to the visitor, and a RESPONSE PANEL 50. wiricb proents question respoiue 
choices to the visitor tod imeracis with the visitor to capture their responses. When its 
storefiesponseO method is adied by QUESTION FRAME 38. QUESTION 40 imerpms the 
5 sute of the RESPONSE PANEL 50 to capture the visitor's response, tnd stores the captured 
response as a string vahie. The question response choices themselves can take any one of a 
number of difierent forms. Examples tnchide: a Ust of mutually exclusive response options, 
presented for instance as option buttons or a single selection list box; a list of non^exdusive 
response options of which the visitor can select one or more, presented, for instance, as check 

1 0 boxes or a multiple selection list box; one or more sliders or the like which allow a response to 
be given on a continuous scale, used, for instance, to respond to a question requesting an 
indiaition of the imponance and/or performance of a particular feature; and one or more text 
fields allowing free hand entry of responses by the visitor, used, for instance, when a question 
asks for the visitoi's comments, or to elicit contact details from the visitor. Specific examples 

15 of these various response choice types are ilhistrated in RGURES 8 to 13, which are 
discussed below. 

Finally, CGI POST 42 functions to send the captured responses in name-value pair 
format to the data grabber servlet 22 of the survey server 14 from its pastp method. 

As already discussed above, the data collection process is iniuated when the visitor 
20 requests a trigger page 12 on a Web site 4 associated with an enabled survey, as they would 
request a page on any other Web site. The trigger page 12 can be any ordinary HTML page 
on the Web site 4, additionally comaining the survey trigger. In the preferred implementation, 
this survey trigger is an applet tag, which the web master can simply insen into the HTML 
source for one or more of their web pages. An exemplary applet tag is shown below: 
25 <applet wdth-5 height=5 

code*"iworks.jvl02.WcbMetrics.AppletLoadcr.class" 
codebase-''http.7/demo. informationworks. nci/wmdemo/classes"> 
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<par8m name-'name" vahie-'demo"> 
</appIct> 

Advantageously, this applet tag. or one like it, can be small (for example only 5 by 5 
pads) and colored to match the Web page 12 it resides on. thus avoiding any disruption of 
5 the appearance of the page. 

To give a better understanding of the data exchange between the various components 
of the system, and the visitor's interaction with the system, a step wise example of the process 
of data collection using \hs invemivc system wll now be described with reference to FIGURE 
3. which illustrutes the complete process, and FIGURES 4 to 17 which show screen shots 
10 taken during the process. The numbered steps described below correspond to the numbered 
anows in FIGURE 3. 

1) The visitor 52. accessing an information web site ("customer Web site") 4 via a 

Web browser 18, requests the trigger page 12 from the Web site 4 (typically the 

Web site of the research commissioner). 
15 2) The Web site 4 returns the requested trigger page 1 2 which coniains an applet tag 

similar to the example given above. 
3) The codebase parameter for the applet tag on the trigger page 12 directs the 

visitor's browser IS to request the survey applet ("WcbMctrics applet) 16 from the 

survey server ("WebMetrics server") 14. 
20 4) The survey applet 16 is returned to the browser 18 by the survey server 14 along 

with related. Java classes. 

5) The visitor's browser 18 executes the survey applet 16. which runs widiin the 
browser 18. 

6) The survey applet 18 requests survey configuration informauon from the survey 
2^ server 14. The configuration information is requested on the basis of the unique 

survey codebase included in the applet tag. 
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7) The applicable survey configuration is returned to the survey applet 16 by the 
survey server 1 4. 

8) The applet 16 executes a validation sequence to determine whether or not to 
present the visitor 52 with an invitation to take the survey. It checks whether or 
not it b supponed under the visitor's browser 18. whether it has been previously 
executed in the currem browser session, and whether or not the visitor's IP 
address has been blocked. If all those tests pass, the survey applet 16 finaUy 
determines whether or not to present a survey invitation to the visitor based on the 
configured invitation frequency, / which is downloaded as pan of the survey 
configuration in step 7. The value of/ is an integer that determines the probability 
that the visitor 52 will be presented with an invitation - the greater the value of/ 
the lower the probability. If/ is equal to 1, then every visitor whose browser and 
IP address pass the initial validation will be presented with an invitation, if the 
validation sequence fails, the applet quietly exits. 

9) If the validation succeeds, the visitor 52 is presented with an invitation 60 to take 
the survey (see RGURE 4), The survey applet 16 posts the fact that an inviution 
60 was presented to the survey server 14 (step 9a). The invitation 60 may or may 
not offer an incentive for the visitor to take the survey. It allows the visitor to 
respond yes or no by clicking the appropriate button 62. 64. 

10) The visitor resporels yes or no. If the response is no. the survey applet quietly 



1 1) If the visitor's response to the inviuilion is yes. the survey applet requests the Java 
classes required to complete the survey from the survey server 14. 

1 2) The requested classes are downloaded to the survey applet 16 from the survey 
server 14. The visitor 52 is presented with a status window for the normally brief 
durmton of the download. Delaying downloads of some of ihe classes up until this 
point reduces the time taken to initially download the survey applet 16, 
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13) The survey questions are requested from the survey serve I4r by the survey applet 
16. 

14) The survey server 14 returns the survey questions to the survey applet 16. The 
visitor 52 is presented with a status window for the normally brief duration of this 
download. Delaying question download up until this point reduces the initial 
download time of the survey applet 16. 

15) The visitor 52 is presented with survey instructions 66 (see RGURE 5) followed 
by a Stan survey icon 68 (see FIGURE 6). The visitor is instructed and encouraged 
to continue with their visit to the information web site 4 before conunuing with the 
survey. 

16) The visitor 52 explores the information web site 4. The stan survey icon S8 
. meanwhile exists as a separate window on the visitor's desktop. As such, it may be 

mininuzed or lowered while the visitor interacts with the browser (see FIGURE 7), 
and effectively "floats' vnxh them as they brxjwse. 

17) The visitor 52 clicks the stan survey icon 58 when ready to begin the survey. 

18) The visitor 52 is presented with the question frame, also in a window separate 
from the browser 18. The question fi^une 70 presents the user with one question at 
a time, examples of which are seen in FIGURES 8 to 13. Each question frame 
displays a banner 72. in which the name of the rcsean:h commissioner can be 
displayed for example, a question panel 74. in which the question text is displayed, 
and a response panel 76 in which the response options are given where applicable 
and in which the visitor enters their response. In addition, the question frame 70 
displays a progress bar 78 indicating the visitor's progress through the survey, and 
a "NEXT- button 80 which the visitor cUcks, having made a response, to move on 
to the next question. Various formats of question response may be used, as 
ilhistrated by FIGURES 8 to 13. FIGURE 8 shows a Im of mutually exclusive 
possible responses alongside respective option buttons 32 that the visitor can dick 
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00 to make their idecrion of response. In RGURE 9 the response pacd displays a 
free fontm text box 84 in which the visitor is invited to type comments. FIGURE 
10 iihistratcs a response panel with two sliders 86. 58 that can be dragged by the 
visitor to their choice of response on the sliding scales shown. In RGURE M. the 
visitor enten their response by dickinj on the desired response in a scroUable. 
single choice list box 90. FIGURE 12 shows an ouunpic of a checkbox list 92. in 
which the user has the option to selea more than one response. Additionally, the 
response panel seen in FIGURE 12 inchides a text box 94 adjacent the "Other 
Internet technology" option, to enter what "other" technology they have. FIGURE 
13 shows a fimher example of a response panel including text boxes 96. here to 
eiicit contact details from the visitor 52. 
!9)The visitor enters a response to each question in turn and after responding clicks 
the "NEXT" button 80 to move to the next question. The response is stored 
locally, for example in the memory or other storage device associated with the 
device on which the visitor's browser is running, and the question frame 70 
prescnu the next question and awaits an answer. This question and answer 
procedure continues until the last quesdon is answered. Step i9a shows mid-point 
progress updates being posted to the survey server 14. If at any time the visitor 
clicks the "NEXT" button 80 without having completed a response, they are 
presented with an appropriate error message. For example, they might be told 
they have not entered a response at all (see FIGURE 14), or that an clement of a 
multi-pan response (see FIGURE 15). In a similar way. in the case of the multi- 
pan "Importance / Performance" response illustrated in FIGURE 10. the visitor is 
also prompted to enter "Importance" before entering "Performance" if they anempt , 
to move the "Performance" slider without having moved the "Importance" slider 
first (see FIGURE 16). 



WO oa/3 1 666 PCTAiS99/n695 
-23- 

20) After the last question is answered, the visitor is presented with a Thank you" 98 
in the question panel 74. and a 'NEXT button 80a to end the survey (see 
RGURE 17). When the user clicks the "NEXT button BOa, the survey servlet 16 
posts all of the question responses to the survey server 14 where they are used to 
update the survey database 6. The survey servlet 14 then exits, leaving the visitor 
52 to continue browsing if they wish. If, however, they return to the trigger page 
1 2, they will not be presented with a fiirthcr survey, because they wiH &i] the 
validation step (step 9 above), After the last question is answcrtd, the counter 
servlet 26 also makes/updates a record of the following attributes (referred to as 
sample incidence infonnation); number of hits to the trigger applet tag; nun^ of 
qualified hits (by browser type), i.e. those which pass the initial validation step; 
number of invitations produced; number of invitadons accepted; and number of 
surveys offered that were conqileted. This combination of features is comparable 
to the type of sample incidence making used to validate traditional market 
research efforts, and is one unique advantage offered by the invention over other 
known on-line research methods. 
Having explained the process by w^ich survey information is collected, the report site 
8 and its interaction with the sur\'ey database 6 to present the designated person with the 
desired reports will now be discussed. As already mentioned above, the report site 8 allows 
designated persons, typically the research commissioner to review the results of an on-going, 
or recently completed survey via a web browser 1 10 (HGURE 1 ). In this way, remote access 
to the real time results is possible from anywhere with Internet access. In the preferred 
implementauon described, the report site 8 is implemented entirely in the Practical Extraction 
and Report Language (PERL) and relies on GGI. It retrieves data from the relational 
database 6 and fonnats it to present the visitor with meaningfijl reports. Using PERL, the 
report site effectively encompasses a report tool 10, because it is able to manipulate the survey 
results stored in the database 6 to present the desired fonnai and focus of the report. 
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A representative session at the repon site 8 is now described with reference to 
nCURES 18 to 28. When a designated user accesses the repon site 8. they arc first 
presented with a login page 1 12 (HGURE 18). Each designated person is given a user name 
and password with which to login, and on entering these in the respective boxes 114. 1 16 on 
the login page and clicking the "Stan" button 118 the designated person is presented with a 
drop down menu 120 (FIGURE 19) of ail active or recently completed surveys that are 
accessible to them (this may be a subset of the complete set of active surveys, some only being 
accessible by other designated users). The designated person selects the survey for which they 
wish to see the resulu and they are then presented with a contents page (HGURE 20) that 
allows them to select one of several reports, namely "Executive Summary". "Counts and 
Frequencies". "Verbatims" or "Crosstabs". or alternatively they can select to export data from 
the database, for example for use with a local spreadsheet or database program. 

By clicking on the first menu item. "Executive Summary" J22. the designated person is 
taken to an Executive Summary contents page (HGURH 2!). where they are presented with 
the choice of three report types. "OvemU". "Site Perfonnance", and "Demognmhics". the 
designated person clicking on the respective button 1 24. 126. 1 28 to be taken to the report of 
their choice. As seen in RGURES 22A and 22B, which respectively show top and bottom 
halves of the relevant report site page, the "Overall" repon gives three general indicators of 
satisfaction of the visiton to the site: overall satisfaction with the site; likelihood of revisiting 
the site; and Ukelihood of recommending the site to a friend or coUeague. In each case, as 
with an of the other reports described below, the results represent an up to date view of 
visitors responses to the survey. The "Overall" repon also pnjvides a link to specific qucsuon 
answers in the 'Verbatims" report (described below) by clicking on the "poor evaluation 
verbatims" button 130. Specifically, this links to the free text answers of the visitors who 
have rated their overall satisfaction as "very dissatisfied" or "somewhat dissatisfied", and who 
have taken the opportunity to comment more fijJly on their reasons for this poor rating. A 
further feature of the "Overall" repon U that it prt)vides the opportunity to view the results by 
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pre-defined cuu or demographic segmem (gender, age. income, Internet experience, 
frequency of visitation to site). To switch the report format to view the results by a particular 
demographic segment, the designated person simply clicks on the respective button 132 at the 
foot of the report (see FIGURE 22B). FIGURES 23A and 238 respectively show extracts 
5 insm the top and bottom halves of an example of one such demographic segment report based 
on income. 

The second report type available from the Executive Summary contents page 
(FIGURE 21) is "Site Performance". Clicking on the respective button 126 takes the 
designated person to the report seen in FIGURES 24 A and 24B. This repon displays the' 
10 visitors' "importance" and "perfonnance' ratings for seven key basic elements relevant to 
most, if not all Web sites. 

The final report accessible from the Executive Summary contents page is the 
'Demographics" repon. This is a full repon of the demographic indicators of the visiton who 
responded to the survey, inchiding indicators such as job occupation, frequency of Internet 
15 usage, age and income. Each indicator is rated as a percentage of total responses. This report 
is seen in its entirety in FIGURES 25 A through D. 

Returning to the main contents page (FIGURE 20), the second option is 'Counts and 
frequetraes". Oicking on this button 1 34 takes the designated person to a repon ^ving a 
question-by-question display of answers to all questions in the survey, with the exception of 
20 free text answers. The aoswen are displayed in an appropriate graphical form, for example 
where the question gave the option of a limited choice of answers, the percentage of each 
■ answer chosen as a percentage of total answers to the question is displayed as a bar graph, 
FIGURE 26 shows an extract from the "Counts and frequencies" report. 

The next report type available on the main contenu page is the "Verbatim" report 136. 
25 As discussed above, in response to some quesaons in the survey, the visitor ts gK-en the 
opportunity to give a free text, open-ended rtaponse. The answers to these questions are 
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nofcd vcrbitim in the survey database, and are accessible through tbe 'Verbatins* report, 
question by quesdoa 

The finaJ report option on the main contents page is the "Crosstabs" rcpon 138. This 
repon provides the designated person with a real-time inieraciive ttxjl, which gives than the 
opportunity to select specific questions to generate a crosstab. The designated person first 
selects the "banner" from a scroll^le list 140 (see FIGURE 27). which may be any one of the 
noD-free tew survey questions, wd then selects the 'stub', which can be any other one of the 
non-free text questions from the survey. The "stub" answers are then displayed crosstabbed 
with the "banner" answers, in both table 142 and bar graph 144 fonn with appropriate 
sutistical notations where applicable (e.g. mean, median, standard deviation and/or standard 
error), as seen in RGLfRES 28 A and 28 B respectively. 

Thus, as ilhistrated by the above, the rcpon site provides a very versatile, interactive 
tool for reviewing the survey results. Furthermore, since tbe rcsulu are updated with the data 
from each newly completed survey, the rcpon site provides real time feed back on the 
performance of the Web site. 

The rcpon site, or an aitemative site accessible via the Internet, may also provide 
access to a survey editor facility. The fundamental component of this facility is essentially a 
data entry tool via which the question data and/or survey configuration data in the survey 
database can be entered and/or amended. Consequently, this facility enables, through means 
known to those of ordinary skill, the designated person(s) to edit an existing survey, or even 
create their own survey, for use with the data collection tool disclosed herein. 

In more detail, the survey editor is a front-end component that enables the entry and 
modification of question content and survey configuration. Using this front-end component, a 
user may enter the following information for each question contained in a survey: question 
text; question type (for example, single response, multiple response, or open ended); response 
text; response dependencies; and order of appearance within the survey. The user can also 
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enter or modify survey specific conftguration data such as question and response color, banner 
and stan-up images, invitation frequency, and [P address blocking. 

Adopting a survey editor such as this will ailoiv someone without skills in the relevant 
computer programming fields to create and modify questionnaires/surveys. In this way, the 
5 connnissioner of the research or other data collection procedure will be able to create ther 
own surveys using this tool. 

The system described above can be implemented on conventional Internet compatible 
hardware. The server side componenu of the data collection tool reside on a survey server or 
servers conneacd to the Internet. The survey database, the rcpon site and the report tool all 
10 reside on a repon server or servers connected to the Interrici. Although in FIGURE I the 
survey server and report server are ilhistrated as being separate servers and may be resident on 
separate physical platfonns, it is also feasible for a single physical platfbim and /or server to 
perform both fiznctions. 

Particularly when using a single physical platform, the currently preferred server 
15 configuration is a Sun SPARC hardware platform, running a Solaris 2.x operating system, but 
other hardware/OS platforms can be used, so long as they meet with the primary requirements 
of offering support for: a robust RDBMS (such as those mentioned below); JDK I ..x, a web 
server that supports the Java serviei API and PERL, or equivalent platforms. JavaWebServer 
I.x is the currently prefened web server, although any web server that supports the Java 
20 scrvlet API couid be used The survey database is preferably an RDBMS such as Sybase 1 1, 
Oracle, Informix or Progress. 

The client to and on which the data collection tool is deployed will typically be a Java 
supporting Web browser such as Microsoft Internet Explorer or Netscape Navigator running 
in a Windows environment, althou^ the invention is also applicable to other browsers and/or 
25 operating systems. The perfonnance requirements for the information server arc dependent 
on the characteristics of the information site itself, rather than the characteristics of the system 
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of the invention which, in the implementation described here, does not use any of the 
resources of the information server, only those of the diem and the survey server. 

While the preferred embodiment of the invention has been illustrated and described, it 
will be appreciated that various changes can be made therein without departing from the spirit 
and scope of the invention. For instance, whilst the system has been described consistently as 
inchiding modules both for collecting data and for reporting the collected data, each of these 
modules may have independem utility in appropriate circumstances. For example, the data 
collection tool may be used as the front end to dau reporting tools other than the inventive 
one described herein. Likewise, the inventive data reporting tool and repon site describe 
herein may be used in conjunction with a database that is populated with data elicited using a ' 
data collection tool other than the inventive tool described herein. 



APPENDIX I 

Exemplary Core Survey f^iyTtjffni 
The following sets forth a set of exemplary survey questions suitable for gathering 
customer feedback regarding the quality of services and information that are being provided 
by an organization doing business on the World Wide Web. Some of the questions arc only to 
be asked if a panicuiar response was made to a preceding question. This is indicated with a 
Boolean statement at the begimring of the question. See for example questions II and 12. 
which are only to be asked if the person taking the survey has selected the first or the third 
response option to question 10. 

qi. Prior to begimiing this survey, about how long did you spend exploring this site? 



Less than a minute. 
One to three minutes. 
Four to ten minutes. 
Over ten minutes. 
Not sure. 
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q2. In your own words, please describe your purpose for visiting this si 
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P,L>fPORTAA:GE£eERgQRarA.X 

In the following quesirons we will present you with attributes thai describe varioui 
charBaensQcs of ■ web site, 

qJ. First, please indicate how imporumi each of these dements arc to vou when you are 



*•* That, we'd like to know your opinion as to how well (JOB DESCRIPTION) 
per;brmed on each of these criteria.. RANDOMIZE ORDER. 



The site's organization makes information easy to fi 



Hot N» Vttr 

imporrani prefereacr Impartial 

I 2 3 4 J 6 7 

O O O O O O O 



o o o o o o o 



99 

o 



_C2 The site contains valuable links. 

Not Na Very 

imp«rual pn&nnc* Imponwit 

I 2 3 J J 5 7 

O O O O O O O 



O O O 0 o o o 



.03 The site contains vahiabte supplementary or educational resources. 



Tiponanr prtfcmin Importini 

1 2 3 4 J 6 7 
O O O O O O O 



o o o o o o o 



_M The latest, most current information is available. 



Hot No Very 

imponint pnftmct important 

1 2 3 4 5 6 7 

O O O O 0 o o 



o o o o o o o 



Very 

99 

o 
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_os The dte loads quickly. 



Hm No V«tt 

InpcnsM pmktcaM Irapertsot 



O O O O O O O 



O O O O O O O 



The site looks attractive. 



Not No Very 

tnpertut prtftmn ImportiBi 

1 2 3 4 J 6 7 

O O O O O O O 



•ttncttro OK 



O O O O O O O 



.07 The site is easy to navigate. 

Not No Very 

tmpomni prvftrtM* iinponuii 

I 2 3 4 J 6 7 

O O O O O O O 



Not Vwy 
MT ai an 

1 2 3 4 5 6 7' 

o o o o o o o 



qJ, Considering all factors, how satisfied are you with this site? 



O Verysausfied 

O Somewhat satisfies 

O Neutral 

O Somewhat dissatisfied 

O Very dissatisfied 



IF qs^Somewhal or Very dissatisfied: Please tell us why you give this Web site a 
poor rating. 
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q7. How likely are you to visit this site again? 

1 O Very likely to visit 

2 O Somewhat likely to visit 

3 O Neither likely nor unlikely to visit 

4 O Somewhat unlikely to visit 
3 O Very unlikely to visit 

qfl. How likely are you to recommend that a friend or colleague visit this site? 

I O Very likely to recommend 

I O Somewhat likely to recommend 

3 O Neither likely nor unlikely to recommend 

4 O Somewhat unlikely to recommend 
i O Very unJikdy to recommend 



<fi. What can we add to make this site more useful to you? 



[ 



q 10. Are you visiting this Web site for business or recreation? 

1 O Business 

2 O Recreation 

3 O Both business and recreation 
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II I. IF " OR "3": Which one of the following best describes your industry? 



Agriculture and Forestry 

Utilities 

Construction 

Manufaaurinn 

Comtiuter and electronics manufacturing 

Software manufacturing 

Retail and wholesale trade 
Investment and insurance service s 
Legal a/c & architeaural services 

Education 

Medical 

Arts and entertainment 

Hotds and restaurants 

Government 
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qiL IF qio--!" OR •*3'':Which depamnem do you work in? 









Financc/accountina 








Adtntnistration 




IT 




Opendaia 




Lesai 




E;22S23 




Manu&cni rin^/produoion 




HR 




Soles 




Purchastnn 




Small business (no dcots) 




Sole pioprictor/owijcr 




Other 
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Hiy How did you most recently get to (JOB DESCRIPTION)? 

1 O Search engine 

2 O Booionark 

3 O Destination site 

* O Unked from another site 



«t4. IF ql3 SEARCH ENGINE: Which search engine did you Unk from? 

1 O Infoseek 

2 O Lycos 

3 O HotBot 
■t O Yahoo 

5 O Excite 

6 O Aita Vista 

7 O Web Crawler 

9S O Other | j 



qlJ, IF qI3 LLNKED FROM ANOTHER SITE: What site did you link from? 



q 1 6. How frequentjy do you access the Inicmet? 

1 0 Houriy 

2 O Daily 

3 OA few times a week 
* OA few tinjes a month 
5 O Less frequently 



<)l 7. Have you ever purchased anything over the Internet? 

1 O Yes 

2 O No 



qi8. How would you evaluate your level of expertise regarding the internet? 

1 O Novice user (new to the Internet) 

2 O Intermediate user (frequent browser, use plug-ins) 

3 O Expert user (programmer, heavy user) 
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ql9 How often have you visited {JOB DESCRIPTION) before today? 



Frequently (more than ten times) 
Once in a while (fewer than ten times) 
Never before 



tlio. Please tell me at what speed you are connecting to the Internet. 



14.4 

28. S 

33.6 

56.6 

ISDN 

T-l 

T-3 

Other. SPECIFY: 
Don't know 



Please indicate which of the following Inleniet technologies you have on your 
computer, 

O Progressive Networks RealAudio, RealPlayer, or RealVideo 
O Shockwave 

□ An Internet phone 

□ VRML phig-in 

□ Microsoft Internet Explorer 3.0 or later 

□ Netscape Navigator 3.0 or !ater 

□ Microsoft NetMeeting 

□ Other Internet technology; | | 

□ None 

□ Don't know 



422. Are you single or married? 



O Stogie 
O Married 



q2j. How many children under the age of 1 S live in your household? 



One 
Two 
Three 

Four or more 
None 



q24. Are you male or female? 



O. Male 
O Female 
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qI3. What is your age? 



Under 18 

\i-14 

25-34 

35-44 

43-54 

55-W 

65 and over 



q26. Please indicate the highest level of education you've completed. 

O Less than high school graduate 

O High school graduate 

O One or two years of college 

O College graduate 

O Graduate school/advanced degree 

q27. Which of the following represenu your total household income? 



Under 510,000 
110,000 to 524,999 
525,000 to 149,999 
550,000 to $74,999 
575,000 to $99,999 
OvTT 5100.000 



Thank you very much! Please complete the following so that we can enter you in the 
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sweepsukes drawins for . 
Name: 

E*miil Address: 
Phone (work); 
Phone (home); 
Addlress: 



tfll We often complete different types of Internet surveys whh respondemj like yourself 
Research subjecu range for studies regarding computer usage, to Internet habits, to 
gathering feedback on new product ideas. Please check here if you'd be willing to have 
us contaa you for other research. 



Yes. contaa me again. 
No. don't contact me again. 



Overail, how would you rate your ocperience on this Internet suivey compared to 
other surveys you may have taken in the past. 



O Much better experience, 
O About the same as others. 
O Much worw experience. 
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1, A computer network based method for collecting and reponing user data to a cfient, 
the method comprising: 
5 deploying a data collection tool to a client on the computer network; 

elidting data from a user accessing the computer network through the client; 
storing the elicited data in a research database; and 
providii^ access to the elicited data to the client. 

10 2. The method recited in claim 1 , wherein deploying a data collection tool is initiated by a 
user visiting a predetermined trigger page. 

3. The method recited in claim 2. wherein predetermined trigger page is one of a plurality 
of predetermined trigger pages. 

15 

4. The method recited in claim 3, wherein deploying a data collection tool further 
comprises: 

detecting the deploying of the data collection toot; and 

controlling the deploying as a function of the results of the detecting. 

20 

5. The method recited in claim 1 . wherein deploying a data collection tool funher 
comprises controlling the deploying as a fiinction of the status of one or more test conditions. 

6. The method recited in claim 5, wherein test conditions comprise at least one of a 
25 response to an invitation to respond to a survey, the quantity of responses to the survey 

relative to a predetermined frequency control function, and the identity of the user. 



7. The method recited in claim 1. wherein storing the elicited data in the research 
database fijnher comprises updating the research database. 
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8. The method recited in claim 1 . further comprising: 

extracting one or more report results from the research database; and 
wherein providing access to the client further comprises presenting one or more of the 
report results to the client. 

5 

9. The method recited in claim 8, wherein extracting one or more repon results from the 
research database further comprises forming one or more repons. 

1 0. The method recited in claim S, wherein presenting one or more of the report results to 
1 0 the client fiirther comprises providing access to one or more of the report results in an 

interactive query format. 

1 1 . The method recited in claim 1 , wherein; 

storing is performed upon an ending of each the eliciting data; and 
15 providing access to elicited data further comprises posting the^ elicited data to a server 

and providing access to the server to the client. 

12. A computer network based method implemented on the Internet for collecting and 
reponing user data to a client over the Internet, the method comprising; 

20 presenting an invitation to the user to take an on-line survey; 

executing the on-ltne survey as a function of the user's response to the invitation, 
executing the on-line survey comprising sequentially presenting a plurality of questions; 
uoring the user's response to each question; 

posting all the responses to a survey server for updating a survey database after all the 
25 questions are presented; 

providing acc es s to all responses to the client upon completion of each instance of 
updating of the survey database. 

1 3. The method recited in claim 12. wherein presenting an invitation to take an on-line 
30 survey fiirther comprises validating the user. 
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14. The method recited in claim 1 3, wherein validating comprises determining one or more 
of the ability of the user's browser to suppon the survey, prior e-xecution of the survey during 
the instant browser session, and a block of the user's internet protocol address. 

5 

15. The method recited in claim 14, wherein validating further comprises determining the 
stattis of a frequency control function. 

1 6. The method recited in claim 1 5. wherein presenting an invitation to the user to take an 
10 on-line survey further comprises presenting a trigger page to a user upon request by the user, 

the trigger page induding a hyperlink to a survey applet. 

17. The method recited in claim 16, fiinhcr comprising updating a database of one or more 
predetermined incidence information data; and 

15 wherein providii^ access to all responses to the client further comprises providing 

access to database of incidence information. 

18. The method recited in claim 17. wherein incidence information dau comprises one or 
more of a quantity of users accessing the hyperiink, a quantity of the users satisfying the 

20 validating; a quantity of invitations presented, a quantity of invitations accepted, and a 
quantity of surveys completed, 

19. The method recited tn claim 12, wherein presenting an bviiatlon to take an on-line 
stnvey fiirther comprises storing an Indication that the presenting an invitation to take an on- 

23 line survey was performed, 

20. The method recited in claim 12, whcrem executing the on-line survey further 
comprises presenting one or more predetermined questions as a function of the user's 
response to a first presented one or more of the questions. 

30 
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21. The method rediett in claim 12. wfaerein posong alt responses to s survey server 
occurs in a sbgle tnnsacuon. 

22. A user dau coUectii^ and reporting sywem impiemented on ihe Iniemei ihe system 
5 comprising; 

a data collection tool for preseming a survey to a user and colleaing the user's 
responses in the form of survey d&u; 

a trigger coupled to data collection tool for tnitiauns the survey; 

a survey dai^ase coupled to the data rollection tool for storing the survey data; 
10 a repon site coupled to the survey database for accessing the survey data; and 

a rcpon tool coupled to the data collection tool for detecting the end of the collecting 
and coupled to the report site for presenting the survey data after detecting the end. 

23. The system recited in claim 22, wherein the report site further comprises a front cod 

1 5 coupled to the survey database and the repon tool for extracting and formatting one or more 
of the survey data. 

24. The system recited in claim 22, fijrther comprising an information site, the trigger 
resident on the information site. 

20 

25. The system recited in claim 22, wherein the data collection tool comprises a server 
side and a client side communicating with the server side via the Internet. 

26. The system recited in claim 2S, wherein the server side comprises; 

2S a survey servlet for extracting data defining a configuration of the survey fiDm the 

survey database; 

a data grabber servlet for accepting a request for one or more of the survey data, 
extracting the one or more survey data, and posting the one or more survey data to the survey 
database; and 
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a question servlet for acceptirtg question text requests, extracting predetermined 
question text from the survey database, and returning the question text. 

27. The system redted in daim 26, wherein the server side further comprises a counter 
5 scrvict for accepting survey pfogre« dau. 

28. The system recited in daim 22. wherein the data collection tool further con^riscs a 
survey applet, the survey applet performing a validation function for determining whether to 
present the survey to the instant usa. 

10 

29. The system recited in daim 22. wherein the trigger comprises a hyperiink to the data 
collection toot. 

30. The system redted in daim 22, further comprising a survey editor tool for editing the 
15 survey. 

31. The system redted in daim 22, wherein the survey dati^ase, report site, and report 
tool reside on one or more report servers connected to the Internet. 

20 32. The system recited in claim 22. wherein the survey database, repon site, and repcn 
tool reside on two or more different senders connected to the Internet 

33. A computer network based method implemented on the Internet for collecting visitor 
data in real-time, the method comprising: 
25 extracting surve\' configuration data defining a configurstion of a survey from a survey 

database, the survey including a set of survey questions; 
presenting the survey to a visitor; 

collecting visitor's responses to all the survey questions in the form of survey data; 
extracting one or more of the survey data; and 
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posting the one or more extracted survey data to the survey database in single 
transaction. 

34. The method recited in daim 33. wherein presenting the survey to a visitor further 
S comprises executing the survey on the visitor's web browser, whereby interface with the 

visitor is provided for collecting the survey data. 

35. The method redted tn daim 33, wherein the survey configuration data is a subset of a 
plurality of configuration data defimng a phirality of survey configurations, each of the survey 

10 configuration inchiding a subset of a plurality of survey questions. 

36. The method recited in claim 3S, wherein the survey configuration data is determined 
as a function of a unique descriptive code. 

IS 37. The method redted in claim 33, wherein presenting the survey to a .visitor further 

comprises determining which of the questions is presented as a function of the user's response 
to an earlier presented question. 

3 8 . The method recited in claim 3 3 , wherein: 
20 collecting further comprises collecting one or more predetermined incidence 

information data; and 

posting further cotnprises updating a database of the inddence information. 

39. The method redted in daim 33. wherein: 

25 collecting further comprises collecting survey progress data; and 

postiitg fiirther comprises posting the survey progress data to the survey database. 

40. A computer network based method implemented on the Internet for reporting 
collected visitor data to a dient in rca]>time, the method comprising: 
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Storing collected data in a research database upon completion of collection of visitor 
data; and 

providing access to the collected data to the dient upon completion of each instance of 
the storing. 

5 

4 i . The method recited in claim 40. further comprising extracting one or more repon 
results from the research database. 

42. The method redted b claim 4 1 , wherdn providing access to the collected data further 
10 comprises providing the report results in an interactive query format. 

43. ' The method redted in daim 4 1 , wherein extracting one or more repon results from 
the research database fiirther comprises storing raw information collected as repon results in 
the database. 

15 

44. The method redted in claim 4 1 , wherdn extracting one or more report results from 
the research database further comprises forming one or more reports. 

43. The method recited in daim 44, wherdn forming one or more reports further 
20 comprises analyzing the report results and forming one or more focused reports. 

46, The method recited in daim 44, wherdn forming one or more reports further 
comprises manipulating the report results and forming one or more focused reports. 

ZS 47. The method redted in daim 40, wherdn providing access to the collected data is 
performed as a fimction of matching one of one or more predetermined access names with a 
correspor^ing predetermined password. 
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48. The method recited tn cl&im 47. wheretn providing acccu to the collected dftU further 
comprises cktermining which one or more of a plurality of report results are sccessibte as a 
fiuiction of the password. 
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aianr Reports ' " ■■ . '^^r^^ 



Home IV^eb^etria lOient Repcrrs |Conaci 



Welcome! 



Enter your losin and paHword in the baxo bdow. Then dick the bunon « the bottom of the page to 
vaEdate your password and eater the rqwrttng she. 

Note: In order to view your reporti, you win ahvays need to enter your login and password via tto 
joUa Tins n«ns thai you wffl not be able to bcofa^ 
boobnaric this page. 



Login: [_ 



Paaiword: L 



\ 





W e b M e t r i c 
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Available Jobs 

The account: twQc can access the fiiOowing jab(s} Gsted bdow. Please pick the job you are intereseied 
in and cSck the button marked 'Stan' to view the on-Hoe rqions. 



'12o 



Last modined January J99S by Wefatnaster 
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Infonoatlon Wcrte Tnfbnnarion Woria DcBoiBaitica' (259 e o m p itn s) 



Table of Contents 



112 



BepartType 



-7 




Last r^fi»** January 5, 1998 by ^yetwromer 



Beport Dcsoiptioa 



Overview of daa 
dsivcd fium the 



Tbe opponmotyto 

^jf iiFTJtw n i><«jh a of 

the 'tfT* 



Ezpon ycur d2t2 for 
usewitfai 

piogmn. 



W e b M e t r i c 



Informatioo Worfai Infenmooii Woria Demongralioa' (259 com^tpa) 



Executive Summary 



Wdcome to the WebMeoic)''*' Exeouive Sunonsiy r epor t. Tfab rqmt pruvidea tn overview of dxn 
derived fiom the WefaMcma™ evahnttoiL Tlis report b or^""'^^ ^ fiiJIovii^ sentocs: 




Report Dqeriptioa 



I^ispisys dst^ on vtsnDTS* ovenD 

Worfcs website, their GkeShood of 
rcvisni^ awd their HV^^'^^'^'^ of 
tfas nts to others. 



Provides dsts oo tbs nnport&ocs 
sod pflrftifiTi'"^" of 



vsnibles tim tre 'drivers* of she 



A detailed loolc tt Triu) is eomii^ to 
the lulluuuuion Worics site sgc, 
ms, gender, levd of Internet 
exp er ti se, and other dsso^itors. 



Last modified -fanutay J, /PW by Webnaster 
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lafonoatioo Works: Infomatioa Worics Demonsgation' 059 c 



Overall Satisfaction 

erpcnence, frequency ofvisaatjon to your lite. . i«=raci 



Coosidcring afl Sicon, how satisfied are you with d 




dick on one of these bottons to see overaD indicator! by demograpbie segment 

J I n m iw I III I 



Last m o^"^'^ Jamtary 5, 199S by Webmaster 




How Ekeiy are you to visit Uiis site agim? 



liE 



r 



Very Ekely to visit 



Neither Ekd^ nor mfikejy to vS] 
Somewfaai nnfifcdy to vtpt. 
VeryunGkg}|ni viaL 



JHb 



How IDttly are you to iKonmiend Jai 1 Krad or coltosue viA tha Sffl^ 
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lafonnation Worfa: Tnfbnnation Wories Deaormraiion' (259 eampUm) 

Site Performance 

These graphics provids vaiuaUe infemiation about your website's perfonnance on bzsic cleincntJ. 
Participants rated the importance of each attribute on a 1-7 scale and then bidicated how they permve 
your site's peifoi ounce for each chaiacterisdc A n^ative *gap' (tmportascs " ^r ^ li pofiinnance) 
h "i"3 ^ ^ opportunity ftjr improvaneat, A positive *gap' suggests thai your she has <tiifpfw< visrarr' 
" "ji^fTTfinfT* ttjjj cntena. 

Data fin- the attributej bdow are ordered from the most m least important criteria. Mean imponancs 
and per&xmance ratings appear to the right of each graph. 

q01.1/qOU: Fiis, please mrit r afr how importam each of these f^i-rt^^ are to you when you are 
^toe.^T^'w^fike to know your opinion as to how wefl 'Infinmaiicn Works Demonstration' 



Site contains vahiable links. 



Site conraini vahiable • 
supplementary or educational 




t 2 3 4 3 • 7 

* Note: Only respondents who indicated both an tmportaoce and p er f o nu anc? rating are inchidfri in 
t h*? analysis. 



Last modified yanuoFy/, 1998 bv Webmaster 



Attribute 



La tes t, most current 
infiarnation is availabie 



Site's o 



Sift is easy to aa:vigate. 



Site I'^'^T quickiy. 



Tlu site looks attiscave. 



Count 



Gap 
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tafonnatioa Works: T ufijiai ation Woiics Denioostntioa* (259 eortpU^^ 



Demographics 

Tl)c &Qowii^ ^npfaics provide 
-wtio panidpazcd in die survey: 



an vnrious destogrspHc inrficaTnn of the respondcits 



QOLIO: Are you visinng dns Web siis for btisneas 



QOLll: Whidi ons of the fbHowing best 
A^rir^^t^^r^ snd forest^ 



Retail ind wholesale trade 



Other 

Internet service provider 

Web devdoper/content provider 

QOZl: Which best desaibes your job ocoipaboa? 



or recj'eiDon? 
81 

59 I 
38 I 

a-178 

your industry? 
t I 
H I 
9 I 
I 

12 I 
14 I 

2 [ 

I ■ c 

S I 
9 I 

5 [ 

1 c 
I 

2 E 

n I 
« I 
I 

1-119 



1^ 
^ ICJ 

1<a 



1^ 
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Sales 



3 (no 

Sole proprietot^owner 
Other 



16 
3 
11 
3 
I 
3 
3 
2 
3 
6 

11-119 



QPL12: How did you mast recently set va T afixim acoo Worts Damnsostian*? 



QOLU: Which n 



sdidyooCnlcfiom? 



Lycos 

^itBot 

Yahoo 

AttaVisa 
WebCnwler 



15 
-178 

2 
5 
8 
20 
2 



QOLIS: How frequently do you a 

Hourly . 

DaDy 

A fisw dines a week 
Afew omes a nostb 
Lett &equestiy 



31 
10 
-178 



WO 000166* 
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Q0L16: Have you ever purchased anything over the Internet? 
Yes 117 
No 61 

h=l78 

Q0L17: How would you evaluate your level of ecpeittse rejartiing the Isteniet? 
NovicB user 42 ^^tZ]_Z 

Intennediaie user 80 

Expert 56 

n-I78 

Q0L18: How often have you vi sited 'I nf bn n aiion Works Danoostnian* before today? 
Fretpieotiy 71' ^^^^^B 

Ones in a while 61 ^BB^ 

NeverbeSwe 46 HHI^ZZZZIII 

n-178 

Q0119: Please tdl me ai what speed you are connecang to the Intenw. 



14.4 


12 


28J 


25 


33.6 


27 


S6.6 


28 


ISDN 


29 


T-I 


36 


T-3 


13 




8 


Dmrt know 


0 

B-178 


Q0L30: Please indicare which of the StOo 




Progressive Networis RealPlayer 


S2 


Mssumcdia Sbnrfcwavc 


6S 


Aslntsnei phone 


54 


VRML cspiI&Sty 


52 


Miooaoft IE 3.0 or late 


71 


Ncsapo3.0artatiT 


90 




39 




11 


Nose 


8 


Doo% Imow 


8 

n-l78 



~] 203 




PCTAJS99/27695 



QShJl: Are you sin^ or married? 
Sm^ 148 
Married Ul ■■ 

n-259 

QOl.22: How many children under the age of 18 live in your housdiold? 

47 



One 
Two 

Four or more 



QQL22: Are you male or female? 

Male 

Female 

Q01^4: What is your age? 

Under 18 

18-24 

25-34 

34^ 

45-54 

55-64 

65 and over 



59 
53 
21 
79 
n-259 

177 
82 
i»-259 

14 
24 
102 
66 
32 
15 
6 

0-259 



Q9L25: Please m^^anic the highcn level of education youVc coinplrted. 



| , f<i fhan tffg^ <rftnril gntdinT ft 



Oso ortwo yeai3 



Gradose sc ho o t ^dva n c ed degree. 

Qpi^ Whicb of the fbOowing rep r e s e nt s your t 
Under SIQ,O0O 
$10,000 to S24^ 
125,000 to 549^ 
150,000 to S74,999 
$75,000 w $99,999 
Over $100,000 



17 
46 
66 
73 
56 
?-258 

32 
17 
47 
63 
48 
43 
O-250 



2^ 



J2» 



1^ 



25D 



W e b M e t r i c g' 



lolbnualioa ^Voricr In&nsiiion Woilo Dei 



QQLO: About how loos did >du spend ocplonng this she dtirii^ das visit prior to bestnaiss tfais 
(crvey? 

One tti tfares miimtei. 51 [ ZC 

Foot to ten orinutea. 48 Bd "j 2c: 

Over ten minutci, 27 M ~i ^SS 

Nmsure.' 11 1 . | P 

0-178 

QOLS: Caostdeii:^ aQ &cton, how satisfied arc you whh ttns site? 
Very satisfied 44 HUC^ 

Sontenfaat itgfftfrri 55 ^^^^B 

NeutiaL 54 HHZZZ 

Somewhat dissas8&d 17 
Veiy i T ' " ' " '» ^ ** t S [ 

11-178 ' 
^^/.-How Ekdy are you to visit this ste agziii? 

Very Ebely to visit 51 ^ 

Somewhat Bceiy to visit 54 BBB^^^ 



Neiriier fikeiy nor unStely to visit 
Somewhat unGkeiy to visit 
Very unGkriy to visit 



36 
27 
10 
0-178 
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Lsformatfoo Woricr "Infimmnoa Works I 



SH rff s qnfTffnn tar the 



Q01.10; AnToavtBCas 

Q01.U: Whicb onof Oa ftdhwist boi dcjcdba rottr 
Q01.I2: HowdidTOBOQancmdjitttD T a ftwnurini i Woda 
QOLU: WUch mch csfiai ^ JOB Qnk from? 
Q01.15:Bgwfre<pntf ydopa«gatt» Iittetn 
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